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This  report  provides  a critical  introduction  to  cost-benefit 
analysis.  Particular  attention  is  paid  to  the  assumptions 
underlying  such  analyses,  and  the  errors  to  which  they  are 
prone.  A number  of  suggestions  are  raised  for  improving  the 
analytic  process  and  the  contribution  it  makes  to  decision 
making . 

® round  and  Approach 

Proposals  for  large-scale  government  and  private  projects  are 
increasingly  coming  under  the  scrutiny  of  cost-benefit  analysis, 
decision  analysis,  and  risk  assessment.  Frequently,  projects 
are  approved  only  if  it  can  be  shown  that  their  expected  benefits 
outweigh  their  expected  costs  and  that  their  design  affords  a 
specified  level  of  safety. 

This  paper  presents  a critical  overview  of  such  analyses.  It 
discusses  (a)  their  rationale;  (b)  what  determines  their 
acceptabil ity  as  guides  to  decision  making;  (c)  the  problems 
such  analyses  encounter;  (d)  how  they  may  be  misused;  and  (e) 
what  steps  are  needed  to  increase  their  contribution  to  our 
society.  The  discussion  is  illustrated  with  a variety  of 
specific  examples  drawn,  in  particular,  from  the  evaluation  of 

new  technologies  (nuclear  power,  supersonic  transports,  etc.). 

/ 


Findings  and  Implications 

Cost-benefit  analysis  and  related  techniques  have  a critical 
role  in  guiding  decisions,  particularly  those  affecting  large 
segments  of  our  society.  Whatever  flaws  such  analyses  may  have, 
they  are  clearly  superior  to  less  systematic  approaches.  It  is 
important,  however,  for  both  the  analyst  and  the  non-expert 
consumer  of  such  analyses  to  understand  the  errors  to  which 
they  are  prone  in  order  to  maintain  a critical  perspective. 
Indeed,  the  institutionalization  of  such  criticism  may  be  very 
valuable . 

Additional  research  is  needed  to  clarify  psychological 
(subjective)  aspects  of  the  analytic  process  in  order  (a)  to 
reduce  the  errors  and  omissions  made  by  the  analysts  and  (b) 
to  improve  communication  of  the  results  of  analyses  and  the 
assumptions  under  which  they  were  reached  to  decision  makers. 
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HE  ART  OF  COST-BENEFIT  ANALYSIS 


1 . INTRODUCTION 

Consumer  and  environmentalist  pressure  over  the  last 
decade  has  dramatically  opened  the  process  of  technoloqy 
requlation  in  this  country  to  public  scrutiny.  To  some  extent, 
this  openinq  has  consisted  of  burstinq  throuqh  doors  that  were 
already  ajar.  Interested  citizens  now  attend  public  hearinqs 
that  15  years  aqo  would  have  drawn  only  qovernment  requlatrrs 
and  industry  representatives.  Another  aspect  cf  the  chanqe  is 
the  emerqence  of  new  forms  of  technoloqy  manaqement,  the  most 
visible  of  which  are  detailed  analyses  of  the  anticipated  impact 
of  proposed  developments.  Risk  assessment,  cost-benefit 
analysis,  decision  analysis,  and  the  environmental  impact 
statement  are  amonq  the  qeneric  names  of  such  analyses.''  Some 
of  the  better  known  examples  are  the  Rasmussen  study  of  nuclear 
power  plant  safety  (Atomic  Enerqy  Commission,  1975),  the  National 
Academy  of  Science  (1975a)  study  of  the  impact  of  supersonic 
transports  (SST'sl  on  the  stratosphere,  and  the  Stanford  Research 
Institute  study  of  the  effects  of  seedinq  hurricanes  to  reduce 
their  intensity  (Howard,  Matheson,  & North,  1972).  The 
preparation  of  such  ana^’yses  has  become  a qrowth  industry,  as 
qovernment  aqencies  comply  with  the  National  Environmental  Policy 
Act  of  1969  requirinq  impact  analyses  for  all  major  federal 
actions  siqni f icantly  affectinq  the  quality  of  the  environment. 

These  analyses  are  tools  for  requlatory  openness  because 
they  force  the  parties  concerned  to  make  explicit  evaluations 
of  the  risks  and  benefits  to  be  expected  from  technol oq i ca 1 
enterprises.  The  assumptions  on  which  these  analyses  are  based, 
and  the  numbers  used  to  derive  summary  cost-benefit  estimates, 


must  also  be  open  to  public  scrutiny.  The  criticism  which 
greeted  the  initial  draft  of  the  Rasmussen  Report  (Atomic  Energy 
Commission,  1974)  and  the  chanqes  made  in  the  final  draft  are 
good  examples  of  how  the  public  can  challenge  these  numbers  and 
assumptions  and  help  produce  more  adequate  estimates  (Primack, 
1975)  . 


Like  the  technologies  they  are  meant  to  assess,  these 
analytic  techniques  have  both  inherent  limitations  and  potential 
for  misuse.  They  will  increase  the  accessibility  and  sensitivity 
of  the  regulatory  process  to  the  interested  public  only  if  that 
public  understands  the  techniques  and  their  foibles  and  monitors 
the  way  the  analyses  are  performed.  In  addition,  special  efforts 
must  be  taken  to  insure  that  the  techniques  are  used  when 
necessary  and  their  conclusions  heeded.  To  this  end,  the  present 
article  describes  some  of  the  goals  of  cost-benefit  analysis, 
the  problems  encountered  by  attempts  to  perform  such  analyses, 
and  the  ways  in  which  specific  analyses  may  be  led  astray  and 
produce  erroneous  results.  It  ends  with  some  suggestions  about 
how  to  maximize  the  social  benefits  of  cost-benefit  analysis. 
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BASIC  APPROACH 


2 . 

The  rationale  of  cost-benefit  analysis  is  that  when 
considering  a proposed  technology,  we  should  assess  in  advance 
the  costs  and  benefits  to  be  expected  from  its  implementation, 
and  then  adopt  it  only  if  the  anticipated  benefits  outweigh  the 
anticipated  costs. 

The  expected  cost  of  a proiect  is  determined  by  (1) 
enumerating  all  aversive  consequences  that  might  arise  from  its 
implementation  (e.g.,  increased  occupa*- iona  1 hazards);  (2) 
estimating  the  probability  that  each  will  occur;  ( ) estimating 
the  cost  or  loss  to  society  should  each  occur;  (4)  calculating 
the  expected  loss  from  each  possible  consequence  by  multiplying 
the  amount  of  the  loss  by  the  probability  that  it  will  be 
incurred;  and  (5)  computing  the  expected  loss  of  the  entire 
project  by  summing  the  expected  losses  associated  with  the 
various  possible  consequences . An  analogous  procedure  produces 
an  estimate  of  the  overall  expected  benefits.'^ 

Performing  a full-dress  analysis  assumes,  among  othc'r 
things,  that  (1)  all  significant  consequences  can  be  enumerated 
in  advance;  (2)  meaningful  probability,  cost,  and  benefit 
judgments  can  be  produced;  (3)  the  often  disparate  costs  and 
benefits  can  somehow  be  compared  to  one  another;  (4)  people 
really  Itnow  how  they  value  different  consequences  today  and  how 
they  will  value  them  in  the  future;  and  (‘i)  what  people  want, 
or  should  want,  is  to  maximize  the  difference  between  expected 
benefits  and  losses. 
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ACCEPTABILITY 


I . 


A normative  deo i s ion-mak i nq  model,  such  as  oost-benefit 
analysis,  is  useful  only  if  it  is  acceptable  to  those  whom  it  is 
supposed  to  quide.  At  first  alance,  cost-benefit  analysis  does 
not  seem  to  play  favorites.  Although  the  decision  reached  in 
any  specific  analysis  will  depend  on  whose  values  are  assigned 
to  the  various  costs  and  benefits,  the  technique  itself  is 
designed  to  accommodate  anyone's  view  of  what  is  good  and  bad 
for  society.  If  the  results  of  a cost-benefit  analysis  seem  to 
favor  unfairly  one  group  in  society  over  another,  the  problem 
would  appear  to  lie  not  with  the  technique,  but  with  the  way  in 
which  it  is  used. 

There  do,  however,  appear  to  be  a number  of  issues  which 
may  render  the  approach  itself,  as  described  above,  unacceptable 
to  some  members  of  the  public.  One  is  that  cost-benefit  theory 
is  concerned  with  the  total  costs  and  benefits  accruing  to 
society  from  a proiect  and  not  with  their  distribution.  For 
many  proiects,  however,  the  risks  accrue  to  different  people 
than  do  the  benefits.  A mountain  village  may  find  itself 
downstream  from  a dam  constructed  to  provide  electricity  for 
consumers  many  miles  away.  Residents  of  the  Hanford,  Washington, 
area  sit  atop  nuclear  wastes  produced  by  power  plants  of  many 
states.  Users  of  aerosol  products  may  be  increasing  everyone's 
chance  of  getting  skin  cancer  for  some  dubious  benefits. 

The  cost-benefit  analyst  typically  deals  with  this  problem 
by  saying  that  if  a proiect's  benefits  outweigh  its  costs,  then, 
in  principle,  the  losers  could  be  compensated  by  the  gainers. 
Although  attractive  in  theory,  such  compensation  may  be 
exceedingly  difficult  to  carry  out  in  practice.  Often  it  is 
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impossible  even  to  identify  the  losers,  for  ex.5mple,  when  they 
are  members  of  future  qener a t i ons . Even  if  ident i f i cat  ion  is 
possible,  the  costs  or  political  difficulties  involved  in  makinq 
compensatory  payments  may  be  prohibitive  (Graaf,  197S),  Unless 
adequate  payback  mechanisms  can  be  quaranteed,  people  may  have 
little  patience  for  analyses  assessinq  net  benefits  (Portney, 
1973)  . 


Cost-benefit  analysis  is  also  mute  with  reqard  to  the 
distribution  of  wealth  in  society.  Therefore,  a prelect  desiqned 
solely  to  redistribute  a society's  resources  would,  if  analysed, 
be  found  to  be  all  costs  (those  involved  in  the  transfer)  and  no 
benefits  (since  total  wealth  remains  unchanqed).  This  balance 
would  only  chanqe  if  it  were  shown  that  redistribution  itself 
miqht  produce  tanqible  benefits  (e.q.  , reduced  crime;  Dan?iqer 
& Wheeler,  197S),  or  if  equity  itself  were  valued.  People 
dissatisfied  with  social  inequities  may  also  find  cost-benefit 
analysis  unacceptable  because  of  its  heavy  reliance  on  current 
market  prices  (reflectinq  current  economic  ar ranqement s ) for 
evaluating  costs  and  benefits.^ 

Another  issue  is  whether  people  really  do  strive  to 
maximize  expected  net  benefits  in  their  own  decision  makinq. 

There  is  a qood  deal  of  evidence  that  the  basic  cost-benefit 
model  is  not  an  accurate  description  of  how  people  make  decisions 
in  actual  practice  (Slovic,  Fischhoff  & Lichtenstein,  197fi;  1^77). 

One  possible  explanation  is  that  people  try  to  follow  the  model, 
but  the  calculations  and  evaluations  reqiiired  are  too  arduous 
to  implement.  If  this  is  the  case,  then  cost-benefit  analysis 
miqht  be  seen  as  a formalized  procedure  desiqned  to  help  people 
make  the  kinds  of  decisions  they  cannot  reach  unassisted.  On 
the  other  hand,  people  may  be  tryinq  to  do  something  quite 


For 


different  from  that  which  is  prescribed  by  the  model, 
example,  perhaps  they  are  most  interested  in  makinq  decisions 
that  are  readily  explained  to  themselves  and  ethers  in  common 
sense  terms.  It  may  be  easier  to  live  with  a qood  iust if icat ion 
(e.q.,  "that’s  the  way  we've  always  done  it")  than  with  the 
dictates  of  a complex  and  perhaps  unintuitive  model. 


APPLICABILITY 


4 . 

Assuminq  that  wo  want  a cost-benofit  analysis,  we  most 
still  ask  whether  it  can  be  perforrru'd  in  any  qiven  situation. 
Specifically,  can  we  do  a qood  enouqh  iob  cf  enumeratinq 
consequences  and  estimatinq  probabilities  and  values  to  justify 
the  enterprise?  The  best  way  to  answer  this  question  would 
seem  to  be  by  considerinq  some  of  the  difficulties  encountered 
in  makinq  such  iudqments.  We  will  consider,  in  order,  the 
enumeration  of  consequences , the  iudqment  of  probability  and, 
finally,  the  judqment  of  value  or  utility. 

4.1  Enumeration  of  Consequences 

In  order  to  list  all  possible  consequences , analysts  must 
consider  not  only  the  performance  of  individual  components  in 
the  system  they  are  studyinq  but  also  interdependencies  between 
those  components  and  the  way  the  system  itself  interacts  with 
the  surroundinq  human  and  physical  environment.  As  one 
indication  of  the  level  of  complexity  that  may  be  encountered, 
some  of  the  analyses  produced  in  the  Rasmussen  study  were  so 
larqe  that  they  exceeded  the  capabilities  of  the  computer  proqram 
designed  for  the  study  and  had  to  be  analyzed  by  hand  (Weatherwax, 
1975)  . To  reduce  these  problems  to  manageable  size  and 
comprehensibility,  a sophisticated  technology  of  reliability 
assessment  (Coates,  1974;  Green  & Bourne,  1972)  has  been 
developed  in  recent  years. 

4.1.1  Tree  Analysis . Two  key  tools  in  this  technology  are 
fault-tree  and  event-tree  analysis.  Each  uses  a tree  structure 
to  show  the  interrelations  between  the  components  of  the 
operating  system.  A typical  branch  point  might  have  a safety 
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j system  either  operating  or  not  operating  in  response  to  an 

■ emergency  situation.  The  "safety  system  fails"  branch  might 

^ lead  in  turn  to  a branch  point  for  "plant  evacuation  alarm 

sounds  or  fails  to  sound".  A "pathway  to  disaster"  is  a chain 
of  events  in  which  the  wrong  branch  is  taken  every  time;  that 
is,  everything  goes  wrong  and  there  is  a major  system  failure. 

The  probability  of  such  a pathway  is  computed  by  considering  the 
probability  of  each  of  its  constituent  failures.  The  risk 
associated  with  a pathway  is  determined  by  multiplying  its 
probability  of  occurrence  by  the  magnitude  of  the  consequences 
should  it  occur.  Adding  the  risks  associated  with  each  of  the 
different  pathways  produces  an  estimate  of  the  riskiness  of  the 
entire  system. 

Event  trees  start  from  a particular  undesired  initiating 
event  (e.g.,  a break  in  a pipe  or  a sudden  stoppage  of  electricity) 
and  project  all  possible  outcomes  of  that  event.  Fault  trees 

start  with  a particular  undesired  final  event  (a  failure  of  the  ; 

system)  and  work  backward  to  identify  the  component  failures  ] 

needed  for  it  to  have  happened.  Essentially,  the  two  techniques 
build  the  tree  from  opposite  ends. 

The  major  danger  in  designing  a fault  or  event  tree  is 
leaving  things  out,  and  thereby  underestimating  the  true  risk. 

The  criticisms  leveled  at  the  Rasmussen  report,  one  of  the  most 
thorough  risk  assessments  done  to  date,  suggest  that  this  danger 
may  be  substantial  (e.g.,  Primack,  1975). 

Several  kinds  of  pathways  seem  to  bo  particularly  prone 
to  omission.  One  type  is  those  pathways  involving  human  error 
or  misbehavior.  The  Rasmussen  study  concluded  that  human- 
initiated  events  were  both  the  greatest  source  of  danger  and  the 


one  most  poorly  understood  (Weatherwax,  197S).  How  ran  we  ever 
be  certain  that  we  have  enumerated  all  of  the  important  and 
imaginative  ways  in  which  we,  the  people  (as  opposed  to  they, 
the  machines) , can  mess  things  up?  Consider  the  Browns  Ferry 
fire,  in  which  the  world's  largest  nuclear  power  plant  came 
close  to  causing  "many  casualties  and  radiation  oontaminat ion  of 
a large  part  of  Alabama  and  Tennessee"  (Comey,  . The  fire 

was  started  by  a technician  checking  for  an  air  leak  with  a 
candle,  in  direct  violation  of  standard  operating  procedures. 

The  fire  got  out  of  control,  in  part,  because  plant  personnel 
were  slow  to  sound  alarms  and  begin  the  reactor  shut-down. 
Disaster  was  averted  finally  when  plant  personnel  "manaqed  to 
jury-rig  pumps  normally  used  to  drive  control  rods  into  the 
reactor  to  pump  water  [to  cool  the  reactor  core)  instead" 
(Business  Week , 1975a).  It  is  a moot  point  whether  such  human 
error  --  or  ingenuity  --  can  ever  be  adequately  enumerated  and 
quantified  for  the  purpose  of  accurate  risk  analysis.  As 
difficult  as  it  may  be  to  quantify  human  frailty,  these  risk- 
analysis  problems  may  be  simple  compared  to  trying  to  pin  a 
number  on  human  malice  (i.e.,  sabotage). 

A second  source  of  omissions  is  failure  to  consider 
unanticipated  changes  in  the  world  in  which  the  technology 
functions  (Coates,  1976;  Hall,  1975).  Risk  assessments  are 
always  predicated  on  some  assumed  constancies  in  the  external 
environment.  These  assumptions  may,  however,  prove  to  he 
erroneous.  For  example,  nuclear  power  plant  design  assumes  the 
availability  of  back-up  electrical  power  sources  should  the 
reactor  fail  and  need  to  be  shut  down.  It  seems  unlikely  that 
any  reactor  fault  tree  designed  before  1965  would  have  incliided 
as  a possibility  the  great  blackout  of  that  year.  Omissit^ns  may 
also  arise  from  assumptions  whose  failure  to  hold  is  much  less 
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surprisinq  than  the  great  blackout,  but  whose  validity  was  simply 
never  questioned.  The  continued  availability  of  properly  trained 
personnel  is  the  sort  of  assumption  that  a tree's  designers  might 
not  even  realize  they  are  making. 

A third  kind  of  omission  arises  from  overcon f idence'  in 
our  scientific  and  technological  knowledge.  An  assumption  of 
most  analyses  is  that  the  system  has  been  designed  correctly 
and  will  work  if  none  of  its  components  fail.  To  the  best  of 
the  designer's  knowledge,  this  is  always  the  case.  But  the 
knowledge  of  even  the  best  engineers  is  limited.  Certainly  it 
is  not  impossible  that  there  are  chemical  and  physical  effects 
yet  to  be  discovered  which  might  threaten  a system's  operation. 

For  example,  despite  the  extensive  study  of  possible  environmental 
problems  that  preceded  its  construction,  the  Alaska  Pipeline 
venture  is  now  threatened  by  the  sudden  and  unforeseen  retreat 
of  the  Columbia  Glacier  near  Valdez  Harbor.  As  the  glacier 
retreats,  it  discharges  large  numbers  of  icebergs,  many  of  them 
undetectable,  in  the  direction  of  the  shipping  lanes  for  tankers 
coming  for  North  Slope  oil  (Carter,  197Sb) . A remarkably  candid 
acknowledgment  of  the  limits  of  one  branch  of  scientific 
knowledge  may  be  found  in  Weisbecker  (1974):  "The  available 

knowledge  concerning  environmental  requirements  of  biological 
communities  cannot  be  used  to  predict  with  any  precision  the 
ecological  effects  of  a WOSA  (snow  enhancement  by  cloud  seeding) 
program"  (p.  xv).^ 

A fourth  type  of  omission  results  from  failure  to  see  how 
the  system  functions  as  a whole.  For  example,  the  rupture  of  a 
liquid  natural  gas  storage  tank  in  Cleveland  in  1944  resulted  in 
128  deaths,  largely  because  no  one  had  realized  the  need  for  a 
dike  to  contain  spillage  (Katz  & West,  1975).  The  DC-10  failed 
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repeatedly  in  its  initial  flights  because  none  of  its  designers 
realized  that  decompress  ion  of  the  cargo  compartment  would 
destroy  vital  parts  of  the  plane's  contrc.'l  system  running  through 
it  (Hohenemser,  1 975).  Green  and  Bournf'  (1  972,  p.  547)  caution 
us  not  to  forget  that  systems  may  well  be  dysf unct iona 1 when 
needed  because  they  are  undergoing  routine  maintenance  and 
testing  or  because  they  have  been  damaged  by  the  testing  process. 

Another  example  of  such  omissions  is  provided  by  a 
National  Academy  of  Sciences  study  of  the  effects  of  thermonucl ear 
war.  The  Academy  panel  decided  that  the  anticipated  reduction  of 
the  earth's  ozone  shield  would  not  imperil  the  survivors'  food 
supply  because  many  crops  could  survive  the  increased  ultraviolet 
radiation.  The  study  failed  to  point  out,  however,  that 
increased  radiation  would  make  it  virtually  impossible  to  work 
in  the  fields  to  raise  those  crops.  "How  was  this  overlooked? 
Because  ...  it  fell  between  the  chinks  of  the  expjert  panels. 

The  botanists  who  considered  the  effects  of  ultraviolet 
radiation  on  plants  didn't  think  to  worry  about  the  workers." 
(Boffey,  1975,  p.  250) . 

A fifth  sort  of  error,  and  one  that  the  Rasmussen  study 
group  took  great  pains  to  avoid,  is  overlooking  what  are  called 
"common  mode  failures".  To  ensure  greater  safety,  many 
technological  systems  are  built  with  a great  deal  of  redundancy. 
Should  one  crucial  part  fail,  there  are  others  designed  either  to 
do  the  same  job  or  to  limit  the  resulting  damage.  In  a nuclear 
power  plant,  for  example,  there  are  many  pipes  carrying  coolant 
to  the  reactor  core.  Should  one  spring  a leak,  others  will  take 
up  its  load  until  it  can  be  replaced.  If  all  fail,  then  the 

reactor  can  shut  down  by  other  means.  Since  the  probability  of 
each  individual  pipe  failing  is  very  small,  the  probainlity  of 
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all  failing  and  the  shut-down  mechanism  failing,  thereby 
creating  a major  disaster,  would  seem  to  be  extremely  small. 

This  reasoning  is  valid  only  if  the  various  components  are 
independent,  that  is,  if  what  causes  one  pipe  to  fail  will  not 
automatically  cause  the  others  to  fail.  "Common  mode  failure" 
occurs  when  the  independence  assumption  does  not  hold.  As  an 
example,  the  discovery  that  a large  set  of  pipes  in  several 
nuclear  plants  were  all  made  from  the  same  batch  of  defective 
steel  (Eugene  Regist<>r  Guard,  1974)  , suggests  that  it  is  not 
implausible  that  several  such  pipes  would  fail  s imu  1 tane'ous  J y . 

At  Brown's  Ferry,  the  same  fire  that  caused  the  core  to  overheat 
also  damaged  the  electrical  system  needed  to  shut  the  plant  down. 
Constructing  a tree  that  considers  all  such  cont ingencies  may  bo 
very  difficult. 

4.2.  Assessment  of  Probabilities 


Assuming  that'  we  have  constructed  the  best  tree  possible, 
we  still  must  estimate  the  probability  associated  with  each  of 
its  links.  Such  estimates  are  most  believable  when  based  on 
extensive  experience.  If  we  have  observed  a particular  piece  of 
machinery  do  its  thing  thousands  of  times,  we  can  normally 
produce  a confident  estimate  of  the  likelihood  that  it  will  fail 
next  time  around.  If  we  are  looking  at  a different,  but  related, 
piece  of  machinery  or  at  the  same  piece  of  machinery  in  a new 
environment  (e.g.,  under  extreme  pressure  or  cold),  we  would  have 
less  confidence  in  the  orignal  estimate.  Our  confidence  would 
also  be  reduced  if  we  had  never  seen  the  entire  piece  of  machinery 
in  operation  but  knew  a great  deal  about  the  reliability  of  its 
components.  If  many  of  these  components  themselves  were  untested, 
our  assessment  problems  would  be  greater  still.  If  the  machinery 
depended  on  its  human  operators  reliably  performing  complicated 
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operational  and  maintenance  procedures,  or  if  sabotaqe  attempts 
were  a real  possibility,  we  might  be  quite  hesitant  about  putting 
much  faith  in  our  estimates. 

A further  complication  arises  from  the  fact  that  the 
systems  whose  riskiness  we  are  most  eager  to  assess  are  those 
with  potentially  the  most  disastrous  consequences.  Such  systems 
are  typically  designed  to  the  highest  standards  of  reliability. 
Unfortunately,  the  more  reliable  an  element  is,  the  larger  the 
sample  of  its  operation  we  must  have  to  estimate  accurately  its 
failure  rate.  "This  means  that  proof  of  low  reliability  may  be 
relatively  easily  obtained,  but  that  proof  of  high  reliability 
may  be  much  more  difficult."  (Green  & Bourne,  1972,  p.  533). 
Thus,  while  we  now  know  that  nuclear  power  is  "fairly  safe", 
whether  it  is  "extremely  safe"  remains  something  of  a mystery. 

To  provide  valid  estimates  in  lieu  of  appropriate 
historical  data,  the  estimators  must  be  experts  in  both  the  topic 
in  question  and  in  the  making  of  probability  estimates.  There 
is  no  guarantee  that  these  two  forms  of  expertise  go  together, 
that  is,  that  those  who  understand  a system  best  are  able  to 
convert  their  knowledge  into  valid  probability  estimates  and  to 
assess  the  quality  of  their  estimates.  In  our  own  work 
(Lichtenstein  & Fischhoff,  1976),  we  have  found  that  people 
who  know  the  most  about  various  topics  are  not  consistently  the 
best  at  expressing  the  likelihood  that  they  are  correct.  It  is 
important  to  know  how  general  this  result  is.  Murphy  and 
Winkler  (1974)  have  found  moderate,  but  systematic  biases  in  the 
probabilistic  predictions  of  experienced  weather  forecasters. 
Performance  was  improved  somewhat  with  intensive  training, 
although  the  training  appeared  not  to  be  readily  transferable 
to  new  tasks  (Winkler,  1975).  Training  has  not  been  tried  with 
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professionals  in  other  fields,  nor  with  people  trying  to  estimate 
the  probabilities  of  extremely  unlikely  events,  the  type  that 
reoccurs  in  risk  assessments.  Indeed,  we  know  little  about  how, 
or  if,  people  distinguish  between  probabilities  such  as  1 in 
100,000  and  1 in  1,000,000.  Psychological  research  is  just 
beginning  to  show  how  to  accommodate  the  fact  that  the  way  in 
which  probabilities  are  elicited  affects  the  estimates  that  are 
produced  (e.g.,  DuCharme,  1970;  Pitz,  1974;  Selvidge,  1975; 

Slovic,  Fischhoff  & Lichtenstein,  1977). 

As  an  example  of  the  sort  of  problems  that  may  be 
encountered  when  making  probability  estimates,  consider 
estimating  the  distribution  of  failure  rates  for  various  machine 
components.  This  distribution  shows  what  proportion  of  the 
components  of  a particular  type  will  fail  once  in  1,000  operations 
(or  hours  of  operation),  once  in  10,000  operations,  once  in 
100,000,  and  so  on.  When  extensive  historical  evidence  is  not 
available,  there  are  a variety  of  judgmental  techniques  for 
estimating  such  distributions.  The  Rasmussen  group  used  a 
variant  of  the  "extreme  fractiles"  method,  asking  their  experts 
to  choose  one  number  so  extreme  that  only  5*5,  of  the  components 
would  have  lower  failure  rates,  and  another  number  so  extreme 
that  only  5%  would  have  higher  failure  rates.  If  these  extreme 
fractiles  are  properly  estimated  and  the  actual  failure  rates 
can  be  measured,  in  nine  cases  out  of  ten  the  observed  failure 
rate  should  fall  between  the  two  estimates.  Fractiles  that  are 
close  together  indicate  that  the  failure  rate  for  the  component 
being  considered  can  be  predicted  with  great  precision. 
Lichtenstein,  Fischhoff  and  Phillips  (1976)  have  reviewed  some 
two  dozen  experiments  testing  the  appropriateness  of  people's 
estimates  of  extreme  fractiles.  These  experiments,  using  a 
variety  of  problems,  a variety  of  ways  of  eliciting  the  extreme 
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fractiles,  and  a variety  of  subiectP  (includinq  storkbrokers , 
weather  forecasters,  and  Harvard  MBA  students)  consistently  found 
that  people's  extreme  fractiles  were  much  too  close  together; 
that  is,  the  true  value  was  much  too  often  either  lower  than  the 
low  fractile  or  higher  than  the  high  fractile.  If  these  results 
may  be  generalized  to  the  estimation  of  fractiles  for  failure 
rate  distributions,  they  suggest  that  the  Rasmussen  report's 
experts  may  have  systematically  overestimated  the  precision 
with  which  they  could  estimate  failure  rates,  which  may  in  turn 
have  led  them  to  be  overconf ident  in  the  precision  of  the 
conclusions  based  on  those  estimates. 


How  good  are  typical  risk  assessments?  Green  and  Bourne 
(1972,  p.  551)  report  that  "in  a typical  example  of  about  RO 
different  system  elements"  assessed  failure  rates  were  within  a 
factor  of  four  of  observed  failure  rates  (i.e.,  between  one- 
fourth  and  four  times  as  large)  for  96s-  of  the  cases,  with  no 
systematic  tendency  to  over-  or  under-estimate.  Similar  results 
are  reported  in  greater  detail  by  Bourne  (1971,  1971),  Eames 

(1966),  and  Hensley  (1968).  Whether  this  degree  of  accuracy  is 
adequate  depends,  of  course,  on  the  magnitude  of  the  possible 
consequences  involved,  the  specific  components  for  which  the 
largest  errors  are  incurred,  and  the  way  in  which  errors  in  the 
estimation  of  failure  rates  for  components  accumulate  to  affect 
the  estimated  failure  rate  for  the  entire  system. 

4 . 3 Val^ues 

The  costs  and  benefits  emerging  from  most  technologies 
are  quite  a varied  lot,  measured  in  units  such  as  dollars, 
aesthetic  value,  and  freedom  to  adapt  new  policies  in  the  future. 
In  order  to  compare  the  expected  costs  and  expected  benefits 
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associated  with  a proposed  technoloqy  --  to  see  if  it's  worth 
our  while  to  adopt  it  --  the  cost-benefit  analyst  must  find  a 
way  to  Gx[iross  these  different  consequences  in  some  common  unit. 
This  is  most  apparent  with  technologies  like  nuclear  power, 
automobi les,  or  the  storage  of  liquid  natural  qas,  in  which  the 
major  expected  benefits  are  measured  in  dollars,  while  the 
major  expected  losses  are  measured  in  reduced  life  expectancy 
and  increased  susceptibility  to  disease  or  violent  accident. 

To  know  whether  or  not  we  want  these  technoloq ies , we  must 
decide  how  much  a human  life  is  worth. 

An  intuitive  response  is  that  there  is  no  way  to  put  a 
value  on  a human  life.  Yet,  in  a sense,  we  do  it  all  the  time. 
Whenever  we  decide  not  to  install  fire  detection  devices  in  our 
homes  or  air  baqs  in  our  cars  or  we  let  a h iqher-pay inq  lob 
draw  us  to  a city  with  a hiqher  crime  rate  or  qreater  earthquake 
danger,  we  are  allowing  some  monetary  reward  to  compensate  us 
for  a slight  reduction  in  survival  probability.  In  a sense,  we 
are  assigning  a value  to  a slice  of  our  own  lives  and  those  of 
our  families.  Although  these  trade-offs  are  seldom  made 
consciously,  for  most  of  us  there  probably  is  some  explicit 
gamble  with  a very  high  prize  for  winning  and  a very  low 
probability  of  losing  on  which  we  would  be  willing  to  stake  our 
lives  (say,  a one  in  a million  chance  to  lose  one's  life  against 
a 999,999  in  1,000,000  chance  to  win  $100,000).  Howard  (1975b) 
has  argued  for  offering  people  a series  of  such  gambles  in  order 
to  determine  the  value  they  place  on  their  lives  and  then  using 
this  figure  where  needed  in  cost-benefit  analyses.  Unless  we 
are  certain  that  such  hypothetical  choices  correspond  to  people' 
real  preferences,  that  the  way  in  which  we  pose  the  gamble  will 
not  affect  its  acceptabi 1 i ty , and  that  people  can  meaningfully 
distinguish  between  probabilities  like  1 in  100,000  and  1 in 


1,000,000  (or  that  our  conclusions  are  unaffected  by  large  errors 
in  estimation),  this  procedure  will  provide  a shaky  basis  for 
important  decisions.^ 

A related  proposal  is  to  look  at  the  values  people  set  on 
their  lives  in  the  implicit  gambles  they  undertake  daily.  For 
example.  Thaler  and  Rosen  (1973)  have  found  that  an  increase  in 
salary  of  about  $200  per  year  was  required  to  induce  men  in  risky 
professions  to  accept  an  increased  annual  probability  of  .001  of 
accidental  death.  From  this,  they  inferred  that  the  value  of 
life,  at  the  margin,  is  equivalent  to  $200,000. 

The  validity  of  this  approach  depends  upon  the  validity 
of  a number  of  not-immediately-obvious  assumptions  upon  which  it 
is  based:  (1)  that  past  preferences  are  valid  indicators  of 

present  and  future  preferences;  (2)  that  people  accurately 
perceive  the  magnitude  of  the  risks  they  accept;  (3)  that  people 
make  decisions  accurately  reflecting  their  true  preferences 
without  being  overwhelmed  by  the  complexity  of  the  decision 
problems  and  therefore  opting  for  sub-optimal  solutions;  and 
(4)  that  the  marketplace  is  responsive  to  people's  desires  and 
provides  them  with  choices  that  allow  them  to  express  their  true 
preferences.  As  a case  study  in  the  tenuousness  of  these 
assumptions,  consider  the  problem  area  of  auto  safety.  Hefore 
the  publication  of  Ralph  Nader's  (1965)  Unsafe  at  Any  Speed,  and 
to  some  extent  today,  most  drivers  )iad  no  idea  of  how  safe  their 
cars  were,  nor  how  safely  they  could  bo  designed,  nor  what  safety 
would  cost,  nor  how  to  go  about  getting  the  auto  makers  to  provide 
them  the  choices  they  wanted  (see  also  Fischer  k,  Kerton,  >975). 

Another  popular  approach  for  setting  a value  on  people's 
lives  is  to  calculate  the  "net  benefit  to  society"  of  having  them 


alive.  This  figure  is  derived  by  subtracting  the  dollar  value 
of  their  lifetime  consumption  from  their  lifetime  earnings. 
However,  as  Bishop  and  Cicchetti  (1973,  , . 112)  not^  , "Ihider  this 
approach  extending  the  lives  of  the  non-wor)<inq  poor,  welfare 
recipients,  and  retirees  is  counted  as  a cost,  not  as  a benefit 

g 

of  a health  program." 

Assessing  the  value  of  a human  life  is  not  the  only  problem 
facing  analysts  in  their  quest  for  a common  measure  for  all  costs 
and  benefits.  Consider  the  difficulties  of  trying  to  measure  the 
value  of  a particular  landscape,  or  of  the  knowledge  that  a 
landscape  is  in  its  original  form  (and  not  reclaimed),  or  of  the 
preservation  of  options  for  future  generations,  or  of  reductions 
in  noise  level  (Bishop  & Cicchetti,  1973;  Fischer,  1974;  Fisher 
k Krutilla,  1973;  Peskin  k Seskin,  1973a).  .Many  cost-benefit 
questions  are  so  complex  that  even  when  dollar  values  can  bo 
assigned  to  different  aspects  of  a project,  it  may  be  extremely 
difficult  to  compute  the  total  value  of  the  project.  Hanke  and 
Gutmanis  (1973,  p.  262)  compared  two  industry-by- industry  studies 
of  the  costs  of  water  pollution  control.  Althougla  derived  only 
a year  after  the  first  study,  the  estimates  from  the  second  (1973) 
study  showed  a mean  absolute  change  per  industry  of  800.  The 
apparent  source  of  these  differences  was  the  number  of 
manufacturing  establishments  included  and  the  distribution  of 
their  sizes. 

Mow  well  are  analysts  able  to  overcome  these  difficulties? 
Tihansky  (1973)  surveyed  200  studies  of  the  benefits  of  water 
pollution  controls  and  found  but  a handful  that  he  felt  wore 
methodologically  valid.  Hanke  and  Gutmanis  (1973)  cited  serious 
shortcomings  in  estimating  the  costs  of  water  pollution  controls, 
the  "easy"  part  of  cost-benefit  analysis  for  water  pollution 
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policy.  According  to  Rowen  (197j),  many  analysts  adopt  the 
easiest  approach  of  all  for  dealing  with  hard-to-measure  costs 
and  benefits:  they  simply  omit  them. 

An  alternate  proposal  that  has  gained  considerable  support 
in  some  quarters  is  the  divide-and-conquer  strategy  of  multi- 
attribute utility  theory  (Edwards,  1971;  Huber,  1974;  Raiffa, 

1968;  von  Winterfeldt  & Fisher,  1975).  Instead  of  trying  to 
assign  a holistic  (dollar)  value  to  a set  of  objects,  followers 
of  this  procedure  first  decide  which  attributes  (or  dimensions) 
of  these  objects  are  most  important  to  them  and  then  evaluate 
each  object  on  each  dimension.  These  judgments  are  then  aggregated 
by  some  formal  algebraic  rule  that  typically  reflects  the  relative 
importance  of  each  attribute  in  order  to  produce  an  overall 
evaluation  (or  utility)  of  each  object.  These  "objects"  could  be 
a set  of  houses  that  one  is  considering  buying  (with  the 
attributes  of  price,  location,  etc . ) or  future  worlds  relying 
on  different  energy  sources  (with  the  attributes  of  pollution, 
interruptability  of  power,  and  so  forth). 

Although  the  multi-attribute  utility  approach  does  not 
solve  the  problem  of  finding  a common  denominator  for  diverse 
attributes,  it  does  tend  both  to  mal^e  the  trade-offs  more  explicit 
and  to  put  attributes  that  are  difficult  to  express  in  dollar 
terms  on  a more  equal  footing.  It  can  also  help  explain  apparent 
inconsistencies  in  people's  preferences.  For  example,  Cohen 
(1974)  has  sarcastically  noted  that  although  the  risks  of  nuclear 
power  appear  to  be  equivalent  to  those  incurred  by  beimj  1/20  of  an 
ounce  overweight,  people  are  much  more  willing  to  accept  the  latter 
risk  than  the  former.  Such  preferences  are  necessarily 
inconsistent  only  if  people  evaluate  their  lives  in  terms  of 
only  one  attribute:  breathing  or  not  breathing.  Consideration 


of  other  attributes,  like  the  quality  of  the  life  that  people  are 
left  with  and  whether  they  must  coexist  with  an  entity  they  find 
utterly  horrific  (nuclear  power),  might  make  these  preferences 
seem  more  reasonable  (see  also,  Pahner,  1976). 

4 . 4 Societal  Gambles 

Implementing  any  new  technology  is  a gamble  of  sorts,  and 
like  other  gambles,  its  attractiveness  depends  on  both  the 
likelihood  af  winning  and  losing  and  how  much  will  bo  won  or 
lost.  Once  wo  have  evaluated  the  risks  involved  with  a proposed 
technology  and  the  benefits  that  may  arise  from  it,  we  must 
decide  if  it  is  worth  our  while. 

Viewing  technological  innovations  as  gambles  may  help 
explain  why  the  controversies  surrounding  them  often  appear  to 
be  irresolvable.  Even  when  people  agree  on  the  risks  and  benefits 
associated  with  a particular  gamble,  there  are  substantial 
individual  differences  in  general  willingness  to  accept  gambles, 
that  is,  differences  in  "risk  aversion"  as  it  is  usually  called 
(Brown,  Kahr  & Peterson,  1974).  Wo  can  speculate  tliat  one  reason 
why  people  argue  so  heatedly  about  the  probabilities  and  values 
associated  with  technological  gambles  is  that,  were  these  issues 
to  be  resolved,  people  would  have  to  confront  the  question  of  the 
sort  of  gambles  that  society  should  take.  Arguing  about  how 
much  risk-aversiveness  is  appropriate  for  society  seems  even  less 
amenable  to  resolution  than  arguing  about  the  facts  of  the  gambles. 

There  appears  to  be  a substantial  aversion  even  to 
acknowledging  that  we  face  gambles  in  our  societal  decision 
making.  Just  prior  to  hearing  a "blue  ribbon"  panel  of  scientists 
report  being  95%  certain  that  cyclamates  do  not  cause  cancer, 
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Food  and  nruq  Administration  Tommissionor  Aloxandpr  M.  Srhmidt 
said,  "I'm  lookinq  for  a cloan  bill  of  health,  not  a wishy-w.ishy , 
iffy  answer  on  cyclamates."  {Euqcrie  Ro'qister  Guard,  lR7f>). 
Recently,  Senator  Muskie  called  for  " ' ope-qrmed ' soient ) st? , who 
do  not  respond  'on  the  one  hand,  the  evidence  is  so,  but  on  the 
other  hand...'"  when  asked  about  the  health  effects  of  pollutants 
(David,  1975).  Analysts  must  be  very  careful  net  to  promise  the 
public  more  certainty  than  their  craft  can  provide.  Such 
promises  can  produce  an  undue  increase  not  only  in  the  {j'Jblic’s 
reliance  on  experts  (and,  perhaps,  in  the  temptat  ic'n  ffT  experts 
to  pass  opinion  off  as  fact,  Kantrowil?,  1975),  but  als«'  in  the 
belief  that  an  analytic  "fix"  can  be  found  that  will  relieve'  us 
of  the  responsibility  of  facing  difficult  societal  decisions. 

According  to  cost-benefit  theory,  decisions  to  accept  or 
reject  qanibles  should  depend  on  those  gambles'  exfK'cti'd  nf>t 
benefit.  Psychologists  and  economists  have  studied  the  g.imbling 
behavior  of  individuals  to  see  if  they  do,  in  fact , adhere  to 
that  criterion.  The  evidence  is  mixed  (e.q.,  Rapofiort  k Wallsten, 

1972) .  Sometimes  pr’ople  are  guided  by  expected  net  benefit;  at 
other  times  they  arc  influenced  by  other  factors,  such  as  the 
way  in  which  gambles  are  presented.  For  example,  L.ichtenstei  n 
and  Slovic  have  found  that  when  asked  how  much  they  are  willin<i 
to  pay  to  participate  in  a gamble,  people  concent  rate  on  how  much 
they  stand  to  win  or  lose:  when  asked  which  of  twr  gambit's  they 
would  prefer,  they  focus  on  the  probability  of  winning  or  losing. 
Working  both  in  psycholoq ica 1 laboratories  and  at  the  Four  Oueons 
Casino  in  Las  Vegas,  they  found  (Lichtenstein  h Slovic,  1971, 

1973)  that  it  is  possible  to  construct  pairs  of  gambles  for  which 
people  prefer  to  play  one,  but  are  willing  to  pay  mor<'  tf^  play 
the  other . 
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If  "simple"  casino-type  gambles  can  lead  to  inconsistent 
behavior,  we  must  expect  even  greater  difficulties  in  evaluating 
technological  gambles  which  are  undertaken  for  society  as  a 
whole,  often  including  future  generations.  Although  such  gambles 
are  far  from  novel  events  (most  decisions  to  go  to  war,  for  example, 
have  fallen  in  this  category),  we  have  no  clear-cut  guidelines 
for  making  such  decisions  in  a democratic  society  (Nash,  Pearce 
& Stanley,  197S;  Zeckhauser,  197S). 

Chauncey  Starr,  a leading  proponent  of  cost -benc’ f 1 1 

analysis,  has  suggested  that  we  use  the  preferences  revealed  in 

past  decisions  to  guide  future  societal  gambling  (Starr,  l^fi^; 

Starr,  Rudman  & Whipple,  lR7(i).  According  to  this  proposal, 

historical  accident  and  fatality  records  reveal  patti-rns  of 

acceptable  risk-benefit  tradeoffs.  Acceptable  risk  for  a tu'w 

technology  would  be  that  level  of  safety  associated  with  ongoing 

activities  having  similar  benefit  to  society.  The  validity  of 

this  proposal  rests  on  much  the  same  assumptions  as  using  current 

market  values  to  determine  the  value  of  life,  and  it  is  prom'  to 
...  9 

the  same  criticism. 

4.5  Criticism  and  Self-Criticism 

It  might  be  tempting  for  non-experts  to  gloat  over  the 
difficulties  that  risk  assessors  face  and  the'  potential  flaws 
in  their  analyses.  One  reason  why  such  gloating  would  be 
misplaced  is  that  we,  the  public,  have  a lot  at  stake  on  how 
well  the  analysts  do.  A second  is  that  we  arc,  after  all, 
equipped  with  the  same  fallible  cognitive  apparatus  that  the 
analysts  have  and,  thus,  could  probably  do  little  better  in  their 
stead.  Society  has  been  gambling  with  people's  lives  for  ye.irs. 
What  cost-benefit  analysis  has  done  is  bring  the  issiu's  underlying 
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theso  gambles  out  in  the  open  to  facilitate  making  clearer,  more 
willful  choices.  To  a large  extent,  once  the  analysts  have  done 
their  best,  the  ball  is  passed  back  to  the  public  or  its  elected 
and  appointed  representatives.  If  the  public  or  its 
representatives  fail  to  understand  the  results  of  these  analyses 
and  thus  fail  to  act  upon  their  impl icat ions , then  their 
sophistication  may  be  for  nought. 

Starr,  Rudman  and  Whipple  (I'l??!)  suggest  a variety  of 
ways  in  which  people's  perceptions  of  risk  are  likely  to  differ 
persistently  from  those  obtained  by  careful  analysis.  For  example 
they  believe  that  the  single  most  important  factor  in  risk 
perception  is  risk  controllability,  an  attribute  which  [leople 
have  often  been  found  to  exaggerate  (Vidmar  & Crinklaw,  1R74)  . 
Kates  (1962)  has  found  that  flood  plain  residents  often  have  very 
inaccurate  ideas  about  the  likelihood  of  floods  in  their  area, 
despite  first-hand  experience  and  extensive'  exposure  to  media 
reports  of  flood  prevalence.  As  a result,  they  often  respond  to 
flood  dangers  in  ways  that  are  not  in  their  best  interest 
(Kunreuther,  1976).  Research  is  needed  to  help  experts  structure 
problems  and  assign  probabilities.  It  is  a i so  needed  to  show 
them  how  to  communicate  the  results  of  their  analyses  to  the 
public  (Slovic,  Fischhoff  & Lichtenstein,  1976). 

For  its  part,  the  public  must  evaluate  the  formal  analyses 
presented  to  it  to  see  whether  they  provide  solutions  to  the 
problems  addressed  or  "merely"  clarify  the  issues  involved;  the' 
public  must  also  evaluate  the  quality  of  its  own  decision-making 
skills  and  take  the  steps  needed  to  acquire  the  skills  it  lacks. 
One  step  forward  would  be  schooling  in  those  aspects  of  cost - 
benefit  analysis  or  decision  analysis  that  do  not  require 
inordinate  amounts  of  specialized  training.  H.  G.  Wells  said 


once  that  "statistical  thinkina  will  one  day  be  as  important  for 
aood  citizenship  as  the  ability  to  read  and  write."  That  day 
seems  to  have  come.  The  public  needs  these  skills  to  influence 
intel 1 iqent ly  the  societal  decisions  that  are  beina  taken  on  its 
behalf,  and  to  re.spond  properly  to  those  problems  when  the 
decision  is  our  own.  At  some  time  in  the  not-too-d istant  future, 
people  livinq  in  earthquake-prone  areas  may  receive  messages  like 
the  followinq:  "There  is  a '"'0%  chance  of  an  earthquake  of 

maqnitude  6.S  and  7.5  alonq  a fault  line  of  10-1“^  miles  centered 
approximately  .50-1  00  miles  south  of  town,  to  occur  1 years  from 
now,  plus  or  minus  6 months."  Will  they  know  how  to  respond  to 
the  qamble  this  messaqe  implies? 
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Understandinq  the  potential  and  limitations  of  cost- 
benefit  analysis  requires  an  understanding  not  only  of  the  basic 
problems  described  above,  but  also  of  the  difficulties  that 
arise  in  actual  practice.  The  problems  tackled  by  cost-benefit 
analysis  are  so  varied  that  no  one  technique  is  adequate  for 
handling  them  all.  Cost-benefit  methodology  provides  the  analyst 
with  a general  approach  to  technology  assessment  and  a bag  of 
tricks  for  measuring  expected  costs  and  benefits  in  individual 
situations.  The  validity  of  any  given  analysis  depends  on  a 
variety  of  specific  factors  such  as  the  messiness  of  the  problem, 
the  skill  of  the  analyst,  the  way  in  which  the  analytic  question 
is  posed,  the  existence  of  appropriate  techniques  in  the  bay  of 
tricks,  and  the  analysts'  ability  to  fashion  new  ones  if  the 
bag  is  empty. 

This  section  considers  possible  problems  with  several 
specific  analyses.  They  were  chosen  not  for  their  being 
particularly  flawed,  but  to  illustrate  difficulties  arising 
with  the  best  of  analyses,  performed  by  the  most  conscientious 
of  analysts,  working  with  limited  resources  and  an  imperfect 
methodology . 

5 . 1 Chemical  Hazards 

In  1974,  the  Environmental  Protection  Agency  commissioned 
a research  team  from  Stanford  Research  Institute  (SRI)  to 
develop  a general  methodology  for  analyzing  the  costs  and 
benefits  associated  with  regulating  hazardous  chemical  wastes 
(Moss,  1975;  Moll,  Baum,  Capener,  Dresch  & Wright,  1975).  The 
SRI  group  chose  as  "exemplary"  noxious  wastes,  asbestos  and 
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cadmium,  a byproduct  of  zinc  smelting  and  tire  manufacturing. 

Their  procedure  was  to  (1)  identify  the  sources  of  asbestos  and 
cadmium  emissions  in  the  U.S.  and  their  place  of  initial 
deposition  (air,  rivers,  solid  wastes),  (2)  characterize  currently 
available  emission-control  technology,  (3)  estimate  the  direct 
costs  of  installing  emission  controls  on  pollution  sources,  (4) 
estimate  the  indirect  costs  of  controls,  primarily  from  loss  of 
world  market  share  due  to  the  increased  cost  of  U.S.  products 
manufactured  under  tight  emission  standards,  and  (5)  estimate 
the  benefits  in  reduced  death  and  illness  that  would  be  obtained 
by  controlling  emissions. 

Performing  this  analysis  required  the  talents  of  a multi- 
disciplinary team  of  experts  in  engineering,  economics,  medicine 
and  decision  analysis.  Assuming  the  competence  of  the  component 
analyses,  the  overall  plan  seems  quite  reasonable.  The 
generality  of  the  rei)ort  appears,  however,  to  be  limited  by  the 
analysts'  policy  of  considering  only  presently  available 
technologies  and  economic  institutions.  This  restriction  is  in 
keeping  both  with  the  SRI  group's  mandate  from  the  Environmental 
Protection  Agency  and  with  the  conservative  analytic  policy  of 
basing  calculations  on  realities,  not  possibilities.  As  a result, 
they  may  have  produced  a "worst-of-a 1 1-poss ible-wor Ids " scenario 
for  evaluating  the  economic  impact  of  pollutions  controls.  It 
assumes  that  cheaper,  more  efficient  pollution  devices  will  not 
be  developed,  that  other  countries  will  not  adopt  similar  controls 
and  increase  their  own  prices,  that  the  U.S.  will  not  restrict 
the  import  of  goods  produced  by  plants  that  do  not  moot  U.S. 
environmental  quality  standards,  that  companies  will  not  reduce 
their  profit  margins  to  maintain  market  share  despite  increased 
production  costs,  and  that  local  areas  will  show  none  of  the 
resilience  needed  to  replace  the  jobs  lost  due  to  reducc'd  mark.et;;. 
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Simplifyinq  assumptions  are  necessary  in  any  analysis. 

For  a report  to  serve  either  the  public  officials  or  the  public 
at  large  which  has  commissioned  it,  these  assumptions  should  bo 
set  out  as  explicitly  as  possible  and  their  effect  on  the  results 
of  the  analyses  explained.  Althouah  listing  assumptions  should 
not  be  too  problematic,  assessing  their  impact  could  necessitate 
a large  number  of  costly  subsidiary  analyses.  An  attractive 
solution  would  be  to  list  the  assumptions  and  let  anyone  who 
questions  them  examine  their  importance.  Given  the  sophistication 
of  analytic  techniques,  however,  few  critics  will  be  in  that 
pos i t ion . 

The  SRI  group's  choice  of  exemplary  pollutants  suggests 
another  way  in  which  analysts'  procedural  decision.-  ■'an  have 
sizable  effects  on  the  results  of  their  efforts.  Of  the  two 
pollutants,  asbestos  is  considerably  bettor  known  to  the  general 
public.  While  cadmium  has  actually  been  judged  a somewhat 
greater  hazard  (Munn,  1973),  it  has  never  stirred  the  sort  of 
controversy  generated  by  Reserve  Mining's  dumping  of  asbestos- 
laden  taconite  tailings  into  Lake  Superior  (Carter,  1974)  or  by 
the  high  incidence  of  lung  cancer  in  asbestos-plant  workers 
(CBS,  1975) . 

Let  us  consider  the  possibility  that  the  SRI  group  had, 
say,  because  of  limited  resources,  been  able  to  analyze  only 
one  of  these  two  pollutants.  At  first  glance,  cadmium  might 
appear  to  be  the  better  choice.  According  to  Moll  (1975),  just 
as  lawyers  have  a saying  that  good  cases  make  bad  laws,  the  task 
of  developing  a general  methodology  for  evaluating  the  expected 
costs  and  benefits  of  pollution  controls  might  be  best  served 
by  choosing  a non-controversial  example.  This  sounds  like  a 
reasonable  rule  of  thumb.  However,  it  is  also  the  case  that 
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the  public  typically  shows  interest  in  only  the  most 
controversial  environmental  issues.  An  analysis  of  asbestos 

I 

certainly  would  elicit  careful  scrutiny  by  both  Researve  Minimj 

and  by  its  opponents.  By  its  scrutiny,  each  side  would  attempt  ' 

to  eliminate  erroneous  material  prejudicial  to  its  position. 

Scrutiny  from  both  sides  is  a valuable  safeguard,  likely  to 
improve  the  quality  of  the  analysis.  If  only  one  side 
scrutinizes,  as  seems  likely  with  non-controversial  pollutants, 
the  resulting  analysis  might  be  unbalanced.  This  particular 
problem  could,  of  course,  arise  with  study  reports  relying  on 
techniques  other  than  cost-benefit  analysis. 

5 . 2 Nuclear  Power 

The  Rasmussen  report  in  its  draft  form  (Atomic  Energy 
Commission,  1974)  was  one  of  the  most  ambitious  and  earnest 
efforts  at  risk  assessment  performed  to  date.  Cogent  criticisms 
of  its  methodology  required  an  impressive  marshalling  of  ; 

talent,  most  notably  by  the  Union  of  Concerned  Scientists 

(Kendall  & Moglewer,  1974;  Kendall,  1975)  and  the  American  > 

Physical  Society's  Study  Group  on  Light  Water  Reactor  Safety 
(1975)  . The  revision  of  the  Rasmussen  report  (Atomic  Energy 
ComiAission,  1975)  is  largely  an  attempt  to  correct  the  problem.s 
found  by  these  critics. 

One  main  criticism  was  that  the  study  had  underestimated 
the  consequences  of  a serious  accident  (should  it  occur), 
particularly  in  terms  of  genetic  defects,  non-fatal  cancer,  and 
groundwater  contamination  (Primack,  1975).  A second  criticism 
was  that  the  probability  of  failure  had  been  underestimated 
(Weatherwax,  1975).  A third,  and  perhaps  the  most  d i scouraq  i n<| , 
was  that  it  is  impossible  to  generate  estimates  of  risk  with  the 
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accuracy  claimed  by  the  report  (Findlayson, 


These  inadequacies  resulted  from  (1)  omissions,  for 

example,  failure  to  consider  the  possibility  of  sabotage  or  of 

procedural  violations  by  plant  personnel  except  under  conditions 

of  stress;  (2)  oversimplifications,  for  example,  the  assumption 

that  failure  rates  are  constant  throughout  the  life  of  a 

component,  whereas  many  components  have  substantially  higher 

failure  rates  at  the  beginning  and  end  of  their  service  1 i f e ; 

(3)  use  of  inappropriate  scientific  evidence,  for  example,  an 

evacuation  model  based  on  experience  with  smaller  numbers  of 

people  in  a smaller  area  and  with  greater  lead-time  than  is 

12 

likely  to  be  available  in  any  real  crisis;  and  (4)  lack  of 
relevant  data,  for  example,  the  absence  of  any  full-scale 
simulation  of  safety  system  operation  in  the  event  of  loss-of- 
coolant  accident. 

It  is  important  to  note  that  these  criticisms  became 
apparent  only  after  intensive  study  of  the  report  by  experts 
from  a variety  of  disciplines.  Much  of  the  criticism  was 
volunteered  by  individuals  concerned  with,  and  oftt^n  hostile  to, 
nuclear  power.  If  such  outside  review  can,  in  fact,  serve  a 
useful  purpose,  it  should  be  inst itut iona 1 i zed  and  dope  1 i t i c i zed 
{see  also,  Rowen,  1973;  Noll,  197fi). 

It  is  also  important  to  note  that  the  great  attention 
spent  on  estimating  the  risks  of  nuclear  power  (three  million 
dollars  for  the  Rasmussen  report,  alone)  has  not  been  matched 
by  a like  effort  to  assess  the  expected  benefits  of  nuclear 
power.  Yet,  at  least  some  of  the  nuclear  power  controversy  may 
be  traced  to  disagreements  about  the  extent  of  these  benefits. 

To  list  but  four  of  the  questions  responsible  for  uncertainties 
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about  the  economics  of  nuclear  power:  "Will  OPEC  fall  apart, 

or  will  the  price  of  oil  for  some  other  reason  drastically 
increase  or  decrease?"'  "Will  a siqnificant  number  of  Americans 
substantially  reduce  their  enerqy  consumpt ion? " : "Will  there  be 

sufficient  capital  to  finance  the^se  enormously  expensive  plants?" 
(Business  Week , 1975b);  "Will  there  be  sufficient  uranium  to 
keep  the  plants  runninq?"  (Day,  1975).  On  the  other  hand, 
earlier  analyses  of  nuclear  power  are  often  faulted  for  havinq 
considered  only  the  benefits  and  iqnorinq  the  risks  (see  also, 
Dyson,  1975).  An  effective  cost-benefit  analysis  must  adequately 
treat  both  sides  of  the  ledqer. 

5 . 3 Earthquakes 

In  1971,  the  Lonq  Beach  City  Council  commissioned  the 
J.  H.  Wiqqins  Company  to  analyze  several  proposed  chanqes  in  the 
city's  buildinq  code,  each  of  which  quaranteed  diffc'rent  decrees 
of  protect  ion  aqainst  earthquake  damaqe  at  some  price.  As 
reported  by  Wiqqins  ( 1972  , 1 973),  this  proic’ct  made  a remarkable 
effort  to  involve  the  public  in  the  process  of  pro[jarinq  the 
report  and  to  make  the  final  recommendat ions  comprehensible  to 
that  public. 

Particularly  notable  was  the  analysts'  realization  that 
people  have  difficulty  understand i nq  very  low  probabilities  in  a 
meaninqful  and  appropriate  way.  As  a result,  they  used  a 
technique  which  compared  the  risks  associated  with  the  various 
possible  buildinq  codes  with  those  associated  with  natural 
hazards.  Althouqh  this  technique  miqht  be  useful  in  many 
situations,  difficulties  can  arise  from  tryinq  to  implement  it 
without  a thorouqh  understandinq  of  the  coqnitive  apparatus  of 
the  people  makinq  the  iudqments.  For  example,  Hewitt  and  Burton 
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(1970;  cited  by  Burton,  Kates  & White,  in  press)  had  residents 
of  London,  Ontario,  judqe  the  probability  of  various  natural 
hazards.  Their  results  showed  that  while  people's  perceptions 
of  hurricanes  and  tornadoes  were  quite  accurate,  they  typically 
overestimated  the  probability  of  floods;  for  ice  storms,  they 
were  spl  i t between  under-  and  overestimators.  Thus,  the  natural 
hazard  chosen  as  a reference  risk  can  seriously  bias,  in  either 
direction,  people's  perceptions  of  the  building  code  risk  they 
are  asked  to  evaluate. 
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Misuse 


All  of  the  problems  discussed  above  must  be  considered 

honest  mistakes,  if  mistakes  at  all.  It  would  be  naive,  however, 

to  assume  that  all  parts  of  analyses  are  performed  to  the  best 

of  the  analysts'  abilities.  Certainly,  it  is  possible  to  bias 

the  results  of  an  analysis  in  a variety  of  fashions.  Some  ways 

like  "a  tendency  of  the  analyst  to  concentra te  on  those  aspects 

of  the  problem  that  are  easier  to  treat"  (Committee  on  Public 

Engineering  Policy,  1972,  p.  14)  or  a tendency  of  the  analyst  to 

have  too  much  faith  in  his  product  and,  therefore,  to  oversell 

it  (Milch,  1976;  Strauch,  1975)  are  fairly  innocent.  Other  ways 

are  more  devious,  as  when  experts  submit  scientific  evidence  of 

low  quality  or  play  "numbers  games"  to  convince  the  public  that 

what  it  wants  is  what  the  analysts  want  it  to  want  (Boffey,  1976; 

Green,  1975;  Kantrowitz,  1975;  Peskin  & Seskin,  1973b;  Schindler, 
1 2 

1976) . 


Short  of  deliberately  slanting  their  results,  analysts 
can  mislead  the  public  by  presenting  information  in  a form  that 
is  unusable.  A 17-volume,  9,000-page  Department  of  the  Interior 
study  of  the  impact  of  an  Alaska  gas  pipeline  has  been  called  by 
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a critic  "-a  monument  to  irrelevancy.  Nowhere  in  it  can  one  find 
a succinct  analysis  of  the  choice  that  must  be  made.”  (Carter, 
1975a,  p.  3fi3;  also  Carter,  1975). 

Presenting  information  in  a usable  form  may  require  a 
fairly  deep  understanding  of  the  cognitive  processes  of  the 
intended  audience.  As  a further  example  of  the  cognitive 
problems  raised  above,  consider  the  importance^  of  analysts 
informing  their  readers  about  the  reliability  of  their  estimates. 
There  is,  however,  abundant  evidence  (e.g.,  Gettys,  Kelly  fv 
Peterson,  1973;  Kahneman  & Tverslcy,  1P7^)  that  were  such 
information  provided,  people  would  not  know  how  to  use  it.  In 
particular,  people  seem  to  be  lust  as  confident  making  inferences 
from  hiqhly  unreliable  data  as  from  reliable  data,  rather  than 
less  confident  as  statistical  theory  dictates, 

Liqht's  (1975)  critique  of  a Department  of  Health, 
Education  and  Welfare  report  on  the  Economic  Cost  of  Alcohn^ 

Abuse  and  Alcoholism  provides  a case  study  of  how  reliability 
information  may  be  undervalued  and  even  iqnored . The  report 
estimated  that  the  economic  cost  associated  with  the  misust*  of 
alcohol  was  $25  billion  a year,  but  hedqed  this  conclusion 
greatly  with  references  to  the  tenuousness  of  some  of  the 
assumptions  that  had  to  be  made  to  complete  the  study.  In  the 
HEW  press  conference  reportinq  the  study,  and  in  its  subsequent 
citation  by  public  figures  like  President  Ford,  this  "admirable 
restraint"  was  absent,  with  the  $25  billion  figure  acquiring  the 
status  of  authoritative  fact.^^ 

Finally,  as  with  other  aspects  of  public  life,  there  is 
the  possibility  of  deliberate  deception.  With  cost-benefit 
analysis,  such  intentions  could  be  manifested  in  shaping  the 
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analyst's  mandate  to  avoid  particular  issues,  obscurinq  loopholes 
in  proposed  leqislation,  hiding  assumptions,  suppressing  the 
results  of  analyses,  or  simply  being  unimaginative.  Rowen 
has  noted  that  "Dominant  alternatives  are  made,  not  born;  they 
are  usually  crafted  by  designers  who  have  a deep  understanding 
of  the  relevant  production  functions,  have  thought  hard  about 
objectives  and  measures  of  effectiveness,  and  are  able  to  shape 
and  modify  alternatives  until  one  or  more  emerge  as  winners'' 

( 1973,  p.  36R)  . 

A society  or  organization  which  relies  on  cost-benefit 
analyses  to  guide  its  decisions  will  have  to  be  alert  to  instances 
of  misuse  buried  in  complex  detail.  Peskin  and  Seskin  warn  that 
"Since  in  most  cost-benefit  analyses,  there  is  considerable  i 

opportunity  to  make  self-serving  assumptions,  it  is  fairly  easy 
to  doctor  the  analyses"  (1973b,  p.  30). 

i 

i 
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CONCl.USION 


6 . 

I believe  that  the  benefits  of  cost-benefit  analysis  can 
substantially  outweiqh  the  costs.  Properly  done  and  used,  it 
can  open  up  the  business  of  technology  assessment  and  regulation 
to  the  public.  It  forces  government  and  industry  to  consider 
societal  costs  and  benefits  in  their  planning  and  to  do  so  in  a 
way  that  allows  the  public  to  criticize  their  analyses.  Those 
who  find  technological  development  and  expansion  repugnant  may 
find  it  hard  to  imagine  an  ally  in  anything  as  technical  as 
these  analyses.  Yet,  it  should  be  noted  that  even  were  a no- 
growth philosophy  to  win  out,  technology  would  still  have  to  be 
monitored,  and  this  seems  to  be  one  of  the  best  ways  of  doing 
it.  Indeed,  in  a no-growth  society,  it  would  take  large 
quantities  of  the  unpolluting  brain  power  invested  in  cost- 
benefit  analyses  to  use  best  the  limited  resources  with  which 
we  would  be  living. 

However,  as  with  any  technology  or  any  component  of  the 
democratic  process,  eternal  vigilance  is  needed  to  make  cost- 
benefit  analysis  serve  its  public  purpose.  Analyses  can  be 
subverted  both  deliberately  and  inadvertently  by  those  who  order 
them,  and  by  those  who  interpret  them.  Many  observers  have 
described,  often  in  fairly  diabolical  terms,  the  col  1 aborat ion 
and  community  of  interests  between  government  regulators  and 
the  industries  they  are  supposed  to  regulate  (Cramton,  1'172; 
Lazarus  & Onek,  1971;  Mineral  King  Valley,  1970;  Mitnick  & Weiss, 
1974;  Noll,  1976;  Of  Birds,  Bees  and  the  FPC,  1967;  Pringle, 

1968;  Sax,  1970).  If  these  descriptions  are  true,  it  is  not 
hard  to  imagine  cost-benefit  analyses  being  quickly  adapted  to 
that  collaboration,  with  regulators  deriving  the  figures  they 
need  to  perform  their  analyses  directly  from  the  regulated 
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industries,  with  little  public  input  (Keatinq,  197'j).  And,  in 
an  atmosphere  of  mistrust,  even  competent,  honest  analyses  arc 
Viewed  with  suspicion.  Some  suaqestions  follow  on  how  to  best 
manage  cost-benefit  analysis. 

Clear  rules  should  be  established  for  what  issues  merit 
cost-benefit  analyses  and  how  their  findings  are  to  be  used. 

For  example,  they  should  cover  not  only  the  scheduling  of  a 
hearing,  but  also  what  O'Leary  (1975)  calls  " underpromoted 
priorities",  projects  without  influential  backers,  but  which 
j may  be  in  the  nation's  interest. 

The  role  of  the  public  in  these  analyses  must  also  be 

formalized.  Obviously,  when  analysts  assign  values  to  the 

positive  and  negative  consequences  of  technologies,  these  values 

must  reflect  the  public's  best  interests,  however  sticky  a notion 

that  is.  Perhaps  less  obviously,  the  public,  in  its  variety, 

must  be  put  in  a position  where  it  can  criticize  the  technical 

aspects  of  the  reports.  This  may  require  not  only  public 

meetings  and  free  circulation  of  cost-benefit  analyses,  but  also 

the  hiring  of  independent  critics  for  each  report.  They  should 

be  paid  as  well  for  the  criticism  as  the  cost-benefit  analysts 

are  paid  for  producing  their  reports,  with  such  funds  being 

1 4 

written  into  every  cost-benefit  analysis  budget. 

{ 

Viewing  cost-benefit  analyses  as  political  instruments 
imposes  a serious  burden  on  decision  analysts.  They  must  not 
only  guarantee  the  technical  correctness  of  their  work,  but  also 
the  validity  of  the  way  in  which  their  research  mandate  is 
formulated  and  the  way  in  which  their  results  arc  used.  They 
must,  in  the  extreme,  be  ready  to  refuse  to  accept  a project 
when  they  feel  that  the  research  question  is  loaded.  For  example. 


* 


I 

i 
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Fay  (197S)  has  identified  what  he  rails  the  "over-rapi ta 1 i zat ion 
rip-off",  in  whirh  an  industry  gets  so  committed  to  a proieot 
that  the  public  can  not  afford  to  let  it  qo  under.  The  analyst 
assigned  to  study  such  a project  must  be  ready  to  show  the 
public  when  and  where  the  original  erroneous  decisions  were  made. 

When  results  they  produce  are  in  danger  of  misinterpretation 
(as  most  complicated  findings  are),  analysts  must  monitor  what 
happens  to  them  once  they  are  released  into  the  public  domain. 

In  controversial  cases,  this  follow-up  may  require  large  amc'unts 
of  unpaid  time.  However,  it  may  be  effort  well  spent  In  ordc-r 
to  maintain  public  confidence,  it  may  be  extremely  important  for 
those  in  the  cost-benefit  business  to  police  their  field 
voluntarily  for  inferior  workmanship. 

Beyond  these  precautions,  analysts  should  do  overytiiing 
in  their  power  to  guarantee  that  the  public  is  not  only  not 
misled,  but  is  actually  properly  informed.  This  means  clarifying 
their  assumptions  and  the  way  they  get  their  fiipires,  worrv'ing 
about  dissemination  in  comprehensible  form  to  the  widest  possible 
audience,  and  making  themselves  available  for  public  diH'iato. 

Perhaps  the  most  important  aspect  of  informing  the  publif' 
is  for  the  analysts  themselves  to  point  out  the  limits  (''f  their 
craft.  Although  such  humility  may  be  painful,  it  protoi'ts  the 
analysts  from  promising  too  much  and  losing  credibility  whenc>vf>r 
their  analyses  prove  flawed.  Humility  will  also  proti'ct  the 
public  from  the  feeling  that  they  must  surrender  responsibility 
for  critical  decisions  to  seemingly  infallible  experts.  A 
public  that  recognizes  these  limits  will  turn  to  the  analyst  not 
for  iron-clad  solutions  to  fjroblems,  but  for  otherwise 
unobtainable  understanding  of  their  intricacies.  If  the  atialysts' 
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best  efforts  at  quant  if icat ion  prove  inadequate,  this  would  be 
seen  as  a sign,  not  of  failure^,  but  of  the  fact  that  some 
questions  of  quality  oannot  be  incorporated  into  analyses,  but 
must  be  studied  in  their  own  right  and  combined  with  the  insights 
produced  by  cost-benefit  studies. 
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7 . NOTES 


^Explication  of  the  differences  between  these  approaches  is 
beyond  the  scope  of  this  paper.  This  term  "cost-benefit  analysis 
is  used  here  to  refer  to  the  broad  spectrum  of  such  techniques. 

2 

The  interested  reader  may  find  more  formal  discussions  of  the 
decision  model  described  here  and  other  related  approaches  in 
Bereano,  Callen,  Kellner,  Olson  & Wengenroth  (1973):  Brown,  Kahr 
& Peterson  (1974);  Coates  (1976);  Gardiner  & Edwards  (in  press); 
Howard  (1975a);  Michan  (1972a);  Peskin  & Seskin  (1973a);  and 
Siebert  & Zaidi  (1975),  as  well  as  in  many  of  the  references 
cited  in  this  paper. 

3 

There  has  been  some  theoretical  work  on  how  to  incorporate 
equity  consideration  in  cost-benefit  analyses  (e.g.,  Uavcman  & 
Weisbrod,  1973;  Hettich,  1976;  Hochman  & Rodgers,  1969;  Mishan, 
1972b;  Raskin,  1975).  This  research  has,  however,  had  little 
impact  on  the  way  analyses  are  performed. 

4 

Nash,  Pearce  & Stanley  (1975)  provide  further  discussion  of  the 
moral  basis  of  cost-benefit  analysis  and  of  analysts'  apparent 
failure  to  understand  the  value-laden  assumptions  of  their  craft. 

^For  example,  with  the  publication  of  the  draft  of  the  report 
bearing  his  name.  Professor  Rasmussen  noted  tliat  the  possible 
presence  of  fundamental  design  errors  in  safety  systems  could 
not  be  predicted  (Gillette,  1974). 

^One  example  of  the  sort  of  surprise  that  may  arise  in  the  wake 
i of  scientific  ignorance  is  provided  by  Philip  Handler  (1973, 

cited  in  Green,  1975a).  In  1938,  on  the  basis  of  research  into 
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causes  of  pellaqra,  he  recommended  that  nicotinic  acid  be  added 
to  corn.  "Pellaqra  disappeared  within  two  years,  in  not  small 
part  because  of  the  forti f icat ion  prcqram.  It  did  not  occur  to 
me  until  some  time  thereafter  that  I had  no  idea  whether  there 
miqht  be  any  ill  effects"  from  the  f ort i f i cat  ion . Two  years 
later,  he  discovered  such  ill  effects  in  rats  fed  larqe  doses 
of  nicotinamide,  the  form  in  which  the  vitamin  occurs  in 
cocnzymes.  For  a further  example,  see  Hammond  Mauqh  (1*^74). 

7 

Zeckhauser  (197h)  reports  that  "Jan  Acton  (1973)  prepared  and 
disseminated  a questionnaire  which  attempted  to  determine  how 
much  individuals  would  pay  for  a mobile  cardiac  unit  that  would 
decrease  the  probability  that  they  would  die  if  thy  had  a heart 
attack.  His  results  suqqested  that  individuals  had  difficulty 
responding  to  the  types  of  questions  he  posed,  thouqh  they 
provided  answers  that  were  not  obviously  unreasonable." 

An  excellent  discussion  of  this  and  other  "Procedures  for 
valuinq  lives"  may  he  found  in  Zeckhauser  (197F).  Also 
Hirshleifer,  Berqstrom  & Rappoport  (1974);  Linnerooth  (197S); 
Zeckhauser  & Shepard  (1976). 

9 

Accordinq  to  the  Committee  on  Principles  of  Decision  Makinq 
for  requlatinq  Chemicals  in  the  Environment,  "The  arqument  for 
relying  on  free  markets  to  allocate  resources  is  based  on  the 
assumption  that  markets  reflect  individual  values;  but  the  very 
existence  of  government  requlation  denies  this  to  the  desired 
deqree."  (National  Academy  of  Sciences,  1976b,  p.  41.)  It  is 
worth  notinq  that  Otway  and  Cohen  (1976)  were  unable  to 
replicate  Starr's  empirical  results. 
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^^This  last  assumption  appears  particularly  \3nreasonable  when 
contrasted  with  the  assumption,  used  elsewhere  in  the  study, 
that  lonq-term  build-up  of  cadmium  in  local  residents  is 
relatively  small  due  to  Americans'  hiah  mobility  (i.e.,  fev; 
people  will  be  close  to  the  plant  long  enough  to  absorb  a lot). 
This  mobility  is  a reflection  of  the  sort  of  responsiveness  to 
changing  economic  conditions  that  presumably  might  provoke  some 
response  to  lost  jobs  other  than  resignation.  See  also  Hanke  J. 
Gutman  is  (1973)  . 

^^The  Brown's  Ferry  experience,  in  which  the  county  civil  defense 
official  was  not  notified  until  two  days  after  the  near-disaster, 
seems  to  cast  further  doubt  on  the  validity  'f  the  model  that 
was  used  (Comey,  1979a,  b) . 

1 2 

In  the  context  of  studies  assessing  the  safety  of  systems  for 
transporting  and  storing  liquid  natural  gas,  Fairley  (1979) 
presents  an  interesting  compendium  of  ways  for  misinterpreting 
and  misrepresenting  accident  statistics. 

^^Glenn  Schweitzer,  Director  of  the  Office  of  Toxic  Substances 
in  the  Environmental  Protection  Agency  has  commented,  "Too  cft('n 
lawyers  and  economists  seize?,  upon  ( sta  t i s t i ca  1 1 y derivu'c]) 
numerical  risk  factors  forgettinci  that  these  experimentally 
derived  estimates  may  in  fact  havt>  a very  shaky  relevance  to  the 
real  world"  (1973,  p.  73).  See  also  L.odgi*  (197A). 

^^Related,  and  somewhat  more  detailed,  proposals  for  independent 
critics  may  be  found  in  Carroll  (1971),  Ege  (1971),  Lazarus  k> 

Onek  (1971),  Mitnick  s,  Weiss  (1  974),  National  Academy  of  Scienc(’-s 
(1975b)  , and  Petak  ( 1 97  3)  . 
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Whatever  their  flaws,  such  analyses  appear  to  have  a critical  role  in  guiding 
social  decision  making.  It  is  important,  however,  for  both  the  analyst  and 
the  non-expert  consumer  of  such  analyses  to  understand  the  errors  to  which 
they  are  prone  in  order  to  maintain  a critical  perspective.  Indeed,  the 
institutionalization  of  such  criticism  is  essential. 

Additional  research  is  needed  to  clarify  psychological  (subjective)  aspects  of 
the  analytic  process  in  order  to  (a)  reduce  the  errors  and  omissions  made  by 
analysts,  and  (b)  help  policy  makers  and  the  public  understand  the  results 
and  the  assumptions  under  which  they  were  reached. 
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